Implementing Gradient Descent Decoding

Many communication channels accept as input binary strings and return output strings of the same length that have been altered in an unpredictable way. To compensate for these “errors”, redundant data is added to messages before they enter the channel. The task of a decoding algorithm is to reconstruct sent message(s) (i.e., to decode) the channel output. There are several critical attributes of a decoding algorithm. The design complexity is a measure of the effort required to design and implement an instance of the algorithm. The implementation cost is a measure of the time and space resources required to decode received sequences once the algorithm is implemented. The apparent accuracy of an algorithm is a measure of its ability to actually identify all most likely sent messages (codewords) for a given received sequence in practice. The proven accuracy is a measure of the algorithm’s certified ability to correctly move from received sequence to nearest codeword without failure of any kind. A decoding algorithm is optimal if it correctly identifies the most likely channel error for each possible received sequence for which there is a unique such error. Error correcting codes have uses beyond communication channels, and different applications have different decoding accuracy requirements. An important challenge is to find methods of proven accuracy that perform optimal decoding and have implementation cost low enough to be practical with codes that include very long words. As the foundations of coding theory developed in the1950s, there arose a method called syndrome decoding that is provably optimal but possibly costly to design and implement. Starting with “turbo” codes and then with low-density parity check (LDPC) codes, iterative decoding has attracted wide interest in the past 15 years. In each case, these terms refer to both encoding and decoding algorithms. The chief advantage of these new methods is their simplicity of decoding implementation. By Shannon’s fundamental work, communication at rates approaching channel capacity requires long codes, which are practical only with low-complexity decoding. Just such communication has been achieved in practice by turbo codes and proven by extensive simulation under strong decoder assumptions for LDPC